{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-----\n",
    "## Predicting Global Supply Chain Outcomes for Essential HIV Medicines using Machine Learning Methods\n",
    "------\n",
    "\n",
    "### Author: Tichakunda Mangono\n",
    "\n",
    "### **Capstone Project, Udacity Machine Learning Engineer Nanodegree, September 2017**\n",
    "\n",
    "\n",
    "\n",
    "- ***Key Question:*** *Can we use procurement transaction data to predict whether a delivery is delayed and estimating the length of the delay*\n",
    "- ***Main Data Source:*** *From The Website: https://data.pepfar.net/additionalData. Procurement transaction data from the Supply Chain Management System (SCMS), administered by the United States Agency for International Development (USAID), provides information on health commodities, prices, and delivery destinations.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Notebook 4: Final Model & Results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### *Chart of Trends in Supply Chain metrics for on-time and in-full delivery rates*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='Data\\chart-declining-supply-chain-performance.png',width=600,height=400>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Project Overview\n",
    "\n",
    "- **Background:** only 19.5M people out of the ~37M people living with HIV are getting the essential medicines they need. Supply of these essential medicines is critical. Recent evidence https://www.devex.com/news/exclusive-documents-reveal-largest-usaid-health-project-in-trouble-90933 suggests that supply chain for major global programs has worsened after recent changes in supply chain managing organizations. See also chart displayed above.  \n",
    "\n",
    "\n",
    "- **Problem Statement:** Such significant supply chain delays in delivery of medicines disrupt treatment and can lead to loss of life and ultimately increases supply chain costs. The goal is to machine learning to determine when and which products are likely to be delayed, as well quantify the extent of the delay. \n",
    "\n",
    "\n",
    "\n",
    "- **Datasets & Inputs:** Publicly available ***supply chain data*** from US, The President;s Emergency Plan for AIDS Relief (PEPFAR) https://data.pepfar.net/additionalData; ***Logistics Performance Index*** data from the World Bank https://lpi.worldbank.org/international/global?sort=asc&order=Infrastructure;   ***Fragile State Index*** data from Fund for Peace data http://fundforpeace.org/fsi/excel/; and finally Factory location and continent from the googlemaps API: http://maps.googleapis.com/maps/api/geocode/json?\n",
    "\n",
    "\n",
    "\n",
    "- **Solution Statement:** A combined \"classification-then-regression\" machine learning algorithm where the classification algorithm predicts whether a particular product will be delayed or not and then the regression algorithm will predict the length of delay on the subset of the data which the classification predicts will be delayed. This mimicks a streamlined, prioritized decision process of a supply chain manager. \n",
    "\n",
    "\n",
    "\n",
    "- **Benchmark Model:** Default SciKit-Learn RandomForestClassifier and RandomForestRegressor will be used as benchmarks/baseline. Several models will then be explored to improve over the benchmark including other ensemble and tree-based models, Support-Vector Machines (SVM), XGBoost.  \n",
    "\n",
    "\n",
    "\n",
    "- **Evaluation Metrics:** Recall and F1-score will be used for classification while R-squared and RMSE will be used for the regression part of the combined model  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Summary of Process so far:\n",
    "\n",
    "#### 1. Data Cleaning\n",
    "- Understand the data descriptions and available fields\n",
    "- Handling Missing Values\n",
    "    - Dosage, Shipping and Line Item Insurance\n",
    "- Investigating Misclassified/micategorized features\n",
    "    - Purchase Quotation date (implied from purchase order date)\n",
    "    - Purchase Order date (imputed from delivery date scheduled)\n",
    "    - Weight - deal with substantial string entries. Calculated average weight per item, molecule or dosage group then multiplied by line item quantity\n",
    "    - Freight Cost - deal with substantial string entries. Same trick as weight, but this time calculated as a proportion of ln_item_weight at the right level. Note bundled vs.unbundled shipments \n",
    "\n",
    "#### 2. Feature Engineering\n",
    "- Feature Extraction\n",
    "    - Dates: year, month, day, weekday, quarter, weekofyear to capture time aspect for purchase order dates, and scheduled delivery date\n",
    "    - Numeric: counts, sums, proportions and measures of central tendency for country-year, factory-year, vendor-year, molecule-year, brand-year\n",
    "    - Categorical: weight captured separately, shipment configuration, freight cost included commoodities, or invoiced separately\n",
    "    - Predicted variables (actual delivery date less scheduled delivery date)\n",
    "    - Time series variations and auto-correlation (lagging, cumulative and rolling stats)\n",
    "- Feature Creation\n",
    "    - Fragility State Index(FSI) for country stability\n",
    "    - Logisitics Performance Index \n",
    "    - Factory location, country and continent (separating and identifying origins vs. destination in all indices)\n",
    "    \n",
    "#### 3. Exploratory Data Analysis & Feature Selection\n",
    "- High corelations in the volumes, quantity and value features\n",
    "- More delays over the weekend? Specific years were worse. Not much signal from the quarter \n",
    "- Significant pairwise correlations\n",
    "- Numerical - some signal from all country , vendor etc. high volumes, trade numbers etc. But vendor festure quite high than others. Some useless ones though. Select those, take away highly correlated ones\n",
    "- Country Fragility and Logistics Indices correlate with Delays\n",
    "- Dimensionality Reduction. PCA first and second cdimensions..\n",
    "- Feature Importances. Several features with small individual impact. Sharing due to correlation\n",
    "\n",
    "\n",
    "#### 5. Pre-processing Pipeline\n",
    "- Pre-processing data pieces, standardscaler, logarithm, Dummifier, labeler, Oversampling techniques\n",
    "\n",
    "#### 6. Model Benchmark\n",
    "- Classification with RandomForestClassifier. Recall=0.264, f1-score- 0.379, total 107 delays correclty found\n",
    "- Regression with RandomForestRegressor: R-squared: 0.876, RMSE:14 days vs. mean of 27 days \n",
    "\n",
    "#### 7. Model Selection\n",
    "- Classification: LinearSVC, SVC, KNeighborsClassifier, LogisticRegressionCV, LogisticRegression, SGDClassifier\n",
    ",BaggingClassifier, ExtraTreesClassifier, RandomForestClassifier, MLPClassifier. Show the chart, and top performer poof\n",
    "    - ExtraTrees, MLP, Random\n",
    "- Regression:Show charts and the numbers; ExtraTrees, MLP, Random"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Final Model and Results\n",
    "#### **Synopsis:**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "A combined “classification-then-regression” machine learning model can avoid the public health \n",
    "and economic costs associated with delayed deliveries of HIV medicines. An ensemble classification \n",
    "algorithm, Extra Trees, is able to detect 1 in 2 delayed item deliveries. This is a significant \n",
    "improvement from a null hypothesis model which would detect only 1 in 9 delayed items and a \n",
    "considerable improvement from benchmarked Random Forest classification algorithm which \n",
    "catches 1 in 3 delayed items. Once delayed items are identified, an Extra Trees regression \n",
    "algorithm can predict the length of delay to within 12 days (RMSE) with an R-Squared of 0.86, \n",
    "an improvement from 16 days (RMSE) and R-Squared of 0.81 with the benchmarked Random Forest regression. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Requirements: software and libraries used\n",
    "- A python file/module \"my_helper_functions.py\" is included in this folder with a set of my own helper functions\n",
    "- The rest of the libraries can be installed using either anaconda or pip distributions\n",
    "- I was running python 3.6.1 on a 64-bit windows system"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Install/Download the following libraries and apis \n",
    "1. python 3.6.1 \n",
    "2. my_helper_functions - provided in this folder. This is my own module of helper functions. It will be required to run most of the code\n",
    "3. pandas\n",
    "4. numpy\n",
    "5. matplotlib\n",
    "6. seaborn\n",
    "7. time\n",
    "8. datetime\n",
    "9. pandas_profiling\n",
    "10. pivottablejs\n",
    "11. missingno\n",
    "12. os\n",
    "13. sklearn\n",
    "14. yellowbrick\n",
    "15. imblearn\n",
    "16. googlemaps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
